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Introduction 



To ensure that all high school graduates are prepared for the opportunities and challenges that await them, 
states have increasingly been focused on aligning their end-of-high school expectations with the demands 
of the real world. In 2005, no state had aligned their expectations with real world demands; now 29 states 
have adopted college- and career-ready standards in English and/or mathematics. And, as standards in 
mathematics have risen, so too have graduation requirements. Currently 19 states and District of Columbia 1 
require students to study mathematics through Algebra II or its equivalent in order to earn a high school 
diploma. These policies are well grounded; advanced mathematics courses, such as Algebra II, improve access 
to postsecondary education, reduce the need for remediation and significantly increase the odds that a 
student will earn a degree. Particularly for disadvantaged students, advanced mathematics coursework in high 
school significantly narrows the equity gap, improving access to and success in college and in future economic 
opportunities. 2 

To assess these raised expectations, a group of American Diploma Project [ADP] network states formed the 
ADP Assessment Consortium. The Consortium created Algebra I and II end-of-course exams that provide an 
honest assessment of whether a student has mastered the content in the course they have just completed 
and is prepared for higher-level mathematics coursework. By design, the tests are challenging. States in 
the Consortium knew that what they were undertaking wouldn’t be easy and that early results would be 
disappointing. But they believed that by setting the bar appropriately high, they could collectively challenge 
themselves, their systems and their schools to improve secondary mathematics education for the benefit of all 
students. 

The ADP Assessment Consortium states have shown that multistate partnerships are not just possible, but 
that they can be a vehicle for driving change. The states have, in fact, been engaged in a voluntary race to the 
top. Even in the face of disappointing first-year results for the ADP Algebra II exam and an economic recession 
that severely impacted state budgets, these states have stayed the course, giving more exams this year than 
last and willingly disclosing the results and the challenges they face. No state alone could do what the 15 
Consortium states have managed together. While there is plenty of hard work ahead, the ADP Assessment 
Consortium states have already demonstrated that they can move faster— and more economically— together 
than they ever could have going it alone. 

Background 

The ADP Assessment Consortium is a group of fifteen states that are part of the American Diploma Project 3 
network— Arizona, Arkansas, Florida, Hawaii, Indiana, Kentucky, Maryland, Massachusetts, Minnesota, New 
Jersey, North Carolina, Ohio, Pennsylvania, Rhode Island and Washington. These states voluntarily banded 
together with the goal of creating high-quality, rigorous mathematics assessments aligned to increased 
expectations, including an assessment that could serve as an indication of readiness for college mathematics. 4 
The resulting end-of-course exams in Algebra I and II represent the largest multistate common assessment 
effort ever undertaken. 

1 To see a full list of the states with college- and career-ready graduation requirements in English and mathematics as well as the specific 
course requirements and the dates that such requirements are effective, please go to 

http://www.achieve.org/files/College&CareerReadyGradReqTable.pdf 
9 http://www.achieve.org/files/BuildingBlocksofSuccess.pdf 

3 For more information about Achieve’s American Diploma Project, go to http://www.achieve.org/files/AboutADP.pdf 

4 A fuller description on the background of the exams and their development can be found at 
http://www.achieve.org/ADPAssessmentConsortium 
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By working together, the states also hoped to create new, high-quality assessments at a lower cost than if 
they were bearing all of the development costs alone. In 2006, the ADP Assessment Consortium states, with 
Ohio serving as the lead, participated in a joint procurement for the development of an Algebra II end-of-course 
assessment. Pearson, after a competitive bidding process, was chosen as the test developer. The first ADP 
Algebra II assessment was administered to nearly 90,000 students in the spring of 2008. Subsequently, the 
Consortium developed Algebra I exam standards and an aligned Algebra I exam, which was administered for 
the first time in 2009. Standard setting [the process of determining achievement-level cut scores] for both 
exams was completed in July 2009. 

The ADP Assessment Consortium states anticipated that early test results would be low but recognized the 
importance of aiming high so that their high school graduates would be prepared for success in college and 
careers. In developing the exams, all participating states were guided by three main purposes: 

■ To develop high-quality exams that would ensure a consistent level of content and rigor in Algebra I and 
II courses within and across states; 5 

■ To improve the quality of curriculum and instruction in Algebra I and II courses, within and across 
states; and 

■ To examine students’ mastery of the content in the course they completed and to provide an indicator 
of students’ readiness for success in mathematics at the next level. With respect to Algebra I, that 
means providing an indicator for readiness in a higher-level mathematics course, such as Algebra 

II. For students completing the Algebra II exam, the exam provides an indicator of readiness for the 
first credit-bearing college mathematics course, typically College Algebra. 

Test Content and Design 

The ADP Algebra I exam assesses algebra skills that will create a strong foundation for the higher-level 
mathematics courses students will take during the remainder of their high school careers. The content on the 
Algebra I exam provides a strong foundation for Algebra II and the ADP Algebra II exam. The ADP Algebra II 
exam assesses the advanced algebra content that is necessary for success in the first credit-bearing college 
mathematics course, typically College Algebra. In this respect, the Algebra II exam is unique and is one of only a 
handful of exams developed with the purpose of signaling to students their readiness for college mathematics 
while they are still in high school. 

The ADP Algebra I and Algebra II exam standards were developed collaboratively by the partner states, based 
in large part on the ADP benchmarks for mathematics. 6 The Algebra I exam standards are rigorous, but 
fundamental in nature, and designed to provide a strong base of knowledge in the basic algebraic content 
necessary for higher-level courses. The Algebra I exam standards are vertically aligned with the Algebra II 
exam standards. The Algebra II exam standards are robust, emphasizing advanced algebra, critical thinking 
and problem solving. The mathematics content assessed and its associated emphasis on each exam are 
described in the following tables. Table 1 describes the four content standards on the Algebra I exam and Table 
2 describes the five core content standards on the Algebra II exam. 7 

5 "Algebra I" and “Algebra II” are common course titles for courses containing the beginning and advanced algebra content normally found 
in such courses. The content tested in the Algebra I and II exams Is also covered in courses with different course titles such as integrat- 
ed high school mathematics. 

B http://www.achieve.org/files/ADPreport_7.pdf 

7 The Algebra I and II exam standards and released items can be seen at http://www.achieve.org/ADPAssessmentConsortium 
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Algebra I Exam Standards The emphasis a particular standard was given on the Algebra I exam is directly 
related to the emphasis the participating states believed that topic should receive in an Algebra I classroom. 
For example, in Algebra I, Operations on Numbers and Expressions is 25 percent of the exam because it is 
considered foundational material that many students will be exposed to for the first time in the course. It 
also serves as an important building block for advanced algebra and is, therefore, included in the Algebra II 
standards as well. Since the bulk of an Algebra I course should focus on Linear Relationships, 35 percent of the 
exam is dedicated to this topic. Non-linear Relationships'^ 20 percent of the Algebra I exam standards. These 
two standards also provide an important foundation for more advanced mathematics including the Algebra 
II exam standards Equations and Inequalities and Polynomial and Rational Functions (See Table 2], It was also 
important to the Consortium states that Data, Statistics and Probab/i/tybe assessed on the exam since the 
content is often included in an Algebra I course and is critical for a well-rounded high school mathematics 
curriculum; 20 percent of the exam is dedicated to this content. 



TABLE 1: ALGEBRA I EXAM STANDARDS 



Standard 


Topics Addressed 


Percentage 
of Total Points 


Operations on Numbers and Expressions 


Operations with numbers and 
algebraic expressions, involving real 
numbers. 


25% 


Linear Relationships 


Linear equations and inequalities and 
systems of linear equations. 


35% 


Non-linear Relationships 


Quadratic and exponential functions 
and the patterns they form. 


20% 


Data, Statistics and Probability 


Data that follow linear trends, 
measures of central tendency and 
determining probabilities. 


20% 



Algebra II Exam Standards Again, the emphasis a particular standard was given on the Algebra II exam 
is directly related to the emphasis the participating states believed the topic should receive in an Algebra II 
classroom. In Algebra II, Operations on Numbers and Expressions comprises only 15 percent of the exam 
because it is considered mostly introductory Algebra II material, some of which should have been covered in 
previous mathematics courses including Algebra I. Building on the foundation laid in Algebra I, Equations and 
Inequalities further explores these concepts at a more advanced level and comprises 20 percent of the exam. 
Polynomials and Rational Functions is the main focus of most Algebra II curricula, and as such is given the 
greatest emphasis on the exam at 30 percent. Exponential Functions was considered important enough to 
make up 20 percent of the test points. Function Operations and inverses, only 15 percent of the total points 
of the Algebra II exam, is the most advanced content in the exam and stretches beyond what is covered in 
some Algebra II courses. The states determined that this content is important and should be included to drive 
curricular improvement and to better prepare students for college mathematics. 
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TABLE 2: ALGEBRA II EXAM STANDARDS 



Standard 


Topics Addressed 


Percentage 
of Total Points 


Operations on Numbers and Expressions 


Operations with numbers and 
algebraic expressions, involving real 
and complex numbers. 


15% 


Equations and Inequalities 


Linear and non-linear equations and 
inequalities, and systems of linear 
equations and inequalities. 


20% 


Polynomial and Rational Functions 


Quadratic functions and higher- 
order polynomial and simple rational 
functions. 


30% 


Exponential Functions 


Exponential functions and basic 
logarithms and their relationship to 
exponents. 


20% 


Function Operations and Inverses 


Combinations and inverses of 
functions. 


15% 



Item Types Both exams include a mix of multiple-choice [worth 1 point], short-answer [worth 2 points] and 
extended-response questions [worth 4 points]. To measure in-depth student understanding of algebraic 
knowledge and skills, the exams include a large number of open-response items that require students to show 
the steps they have taken and to justify their reasoning when solving a problem. At least 30 percent of the 
total points are from the short-answer and extended-response items. These problem solving and reasoning 
skills are critical for higher-level mathematics work, including credit-bearing coursework in college, and are 
highly valued by mathematics educators and postsecondary faculty. 

Calculator Use In developing the content of the exams, the state mathematics experts felt it was necessary 
for students to demonstrate fluency in mathematics both with and without the use of technology. As a 
result, the exam is structured into two sections; one that allows the use of a calculator and one that does not. 
Although not required, the use of a graphing calculator is highly recommended for the calculator section. 

2009 Changes to the Algebra II Exam The Algebra II exam was first administered in Spring 2008. Although 
the test was untimed, it was estimated that test administration would take 45 to 60 minutes per session. 
Feedback from schools and classrooms after the first administration, however, indicated that students took 
longer than expected to complete the test. In response to concerns regarding test length, the states decided 
to reduce the number of items on the Algebra II exam for the Spring 2009 administration, as well as all future 
administrations, and to extend the estimated time per session to 90 minutes. The test remains untimed for 
students. 

Additional Modules The ADP Algebra II end-of-course exam content standards identify content for the core 
exam, as well as content for seven optional modules. The modules were developed to enable growth of the 
Algebra II end-of-course exam beyond the traditional Algebra II curriculum. The modules available are Data 
and Statistics, Probability, Logarithmic Functions, Trigonometric Functions, Matrices, Conic Sections, and 
Sequences and Series. The modules have been developed and field tested but, to date, have not been 
administered in any state. 
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Comparing the ADP Algebra II Assessment to other Exams and International 

Standards 

The ADP Algebra II exam standards were developed by high school and college mathematics 
faculty from the participating states. The exam is challenging, but not unreasonably so. It 
provides an honest measure of how well students have mastered the course content, and how 
well prepared they are to enter and succeed in a college mathematics course. 

The Algebra II exam includes far more advanced algebra than high school exit exams. In 20Q4, 
Achieve conducted a review of graduation assessments, Do Graduation Tests Measure Up? 

A Closer Look at State High School Exit Exams, which included a review of the mathematics 
exams required for graduation in six states, Florida, Maryland, Massachusetts, New Jersey, 
Ohio, and Texas. Looking across these state exams, Achieve found that nearly 60% of the 
mathematics items on high school graduation exams measure pre-algebra skills, whereas only 
16% of the ADP Algebra II exam measures similar content. In contrast, only 15% of the items 
on state high school graduation tests measure advanced algebra content, while nearly 60% of 
the items on the ADP Algebra II exam measure this advanced content. 

The Algebra II exam includes more advanced algebra than college admissions exams. The ACT 
and SAT are comprehensive mathematics admissions tests assessing a mix of topics. Their 
intended purpose is for use in admissions decisions, not to measure the mastery of specific 
course content, especially an advanced course such as Algebra II. While 60% of the ADP 
Algebra II exam measures advanced algebra content, according to ACT the content covered 
by the mathematics portion of the ACT exam includes the following mix: pre-algebra (23%); 
elementary algebra (17%); intermediate algebra (15%); coordinate geometry (15%); plane 
geometry (23%); and trigonometry (7%). The College Board reports that the topics covered 
on the math portion of the SAT exam include numbers and operations; algebra and functions; 
geometry; and statistics, probability and data analysis. Topics from third-year college- 
preparatory math courses (such as Algebra II) were first added to the SAT in 2005. For more 
information about what admissions and placement exams test, including an analysis of the 
mathematics portions of the ACT and SAT exams, go to Aligned Expectations? A Closer Look at 
College Admissions and Placement Tests at www.achieve.org/AlignedExpectations. 

The Algebra II exam standards are internationally competitive. Achieve recently compared 
the Algebra II end-of course exam standards to upper-level secondary mathematics standards 
in eleven countries— Alberta (Canada), China, Chinese Taipei, Finland, Hong Kong, Japan, 

Korea, Malaysia, New Zealand, Singapore and Thailand. The mathematics standards from the 
countries Achieve analyzed are the upper-level secondary standards required for all students 
receiving a high school diploma or equivalent credential. The analyses show that the algebra in 
the standards from other countries is at least as rigorous, if not more, than the ADP Algebra 
II exam standards. In more than half of the countries’ standards analyzed, over 50% of the 
algebra content focused on advanced algebra. 
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Administration of the Spring 2009 ADP Algebra I and Algebra II Exams 



The development of the ADP Algebra I and Algebra II end-of-course exams is an important element in the 
college- and career-ready agenda in the ADP Consortium states. Most of the states in the Consortium are 
still in the process of developing and adopting policies that will govern participation in the exams and how the 
results will ultimately be used. Consistent with this ongoing discussion, it is important to note that there are no 
stakes— no incentives or consequences — for students, schools or districts attached to either ADP Algebra I or 
Algebra II exam results. 

Five states participated in the first administration of the Algebra I exam: Kentucky, Minnesota, New Jersey, 

Ohio and Rhode Island [see Table 3, p. 11] with 33,446 students tested in total. The exam was treated by all five 
ADP Consortium states as an opportunity to pilot the test in a small number of school districts and schools. 
Some consortium states are already planning to phase in broader participation for future test administrations; 
for example, New Jersey is planning to have all students who take Algebra I in the 2009-10 school year take the 
ADP Algebra I end-of-course exam in spring 2010. 

With respect to Algebra II [see Table 4, p. 11] most states, in the second administration year of the exam, 
decided to continue to pilot the exam in a small number of school districts and schools. The notable exceptions 
were three states— Arkansas, Hawaii and Indiana— that required all students taking an Algebra II course in the 
spring of 2009 to take the ADP Algebra II end-of-course exam. These states are all evaluating how a rigorous 
Algebra II assessment can reinforce their college- and career-ready policies. 

For example, Arkansas is one of 19 states that require all students to complete an Algebra II course to 
graduate from high school, starting with the class of 2010. Hawaii, while not currently requiring all students 
to take Algebra II, includes the course as part of its Board Recognition Diploma and is working with both 
postsecondary institutions and employers to create incentives for students to complete the more rigorous, opt 
up curriculum. Hawaii is also, through a robust multimedia campaign, encouraging their high school students 
to "Step Up” to the challenge. Starting with the graduating class of 2011, Indiana’s Core 40 requires at least 
three years of mathematics though Algebra II, or a three-year integrated course sequence covering the same 
curriculum, for graduation. Indiana is also moving to an end-of-course assessment system and considering how 
the ADP Algebra II exam may fit into that system. Again, even in these three states where all students who 
took Algebra II were required to take the exam, there were no stakes attached to the results of the exam for 
students, schools or districts. 

This year, the economic recession and its impact on state budgets had a significant impact on test 
administration and participation for both exams. Interest in the exam remained high, but the ability for states 
to test as many students as they would like was limited by budget realities. For instance, Ohio tested over 
33,000 students in Algebra II in 2008, but was only able to secure funding for 2,000 Algebra I exams and 
nearly 2,500 Algebra II exams in 2009. Many other states that were able to offer the Algebra II exam to all 
districts last year were forced to select districts to participate this year instead. 

Despite the fiscal challenges faced by states, participation in the ADP Algebra II exam increased this year. 
Across the Consortium, 102,936 students took the Algebra II exam in the spring of 2009, up from 88,344 in 
2008. 
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Standard Setting 



The spring 2009 test administrations were important for both exams because results now can be reported 
in terms of performance levels indicating a student’s mastery of the course materials and readiness for 
success in mathematics at the next level. The process of determining the achievement-level cut scores for the 
exams— known as standard setting— was completed in July 2009 and involved representatives from each of 
the Consortium states. 

Algebra I The ADP Algebra I End-of-Course Exam used a traditional item mapping standard setting approach, 
commonly used in state assessment programs. In this procedure cut scores are based on the judgments of a 
standard setting panel, in this case state department mathematics specialists and high school mathematics 
teachers. During a two-day meeting involving multiple rounds of ratings and discussions, panelists made 
judgments to match student performance on the test to achievement level descriptions based on the ADP 
Algebra I exam standards. At the conclusion of the meeting, the judgments of individual panelists were 
combined to produce a set of recommended achievement-level cut scores for the ADP Algebra I end-of-course 
exam: “ Below Basic” "Basic" “ Proficient' or " Advanced. ” For a fuller description of these performance levels, see 
the sidebar on page 8. 

Algebra II Because of its intended use as both evidence of content mastery and an indicator of readiness 
for a first-year credit-bearing college mathematics course, a variety of empirical and judgment-based studies 
conducted over the past year and a half formed the basis of the standard setting process used for the ADP 
Algebra II end-of-course exam. This research served as the main evidence for the panelists’ judgments on the 
placement of the achievement level cut scores. The types of research studies used to inform the standard- 
setting process included the following: 

■ International benchmarking analyses, in which Achieve content specialists compared the content and 
rigor of the ADP Algebra II exam standards with upper-level secondary mathematics standards in 
eleven countries; 

■ Concurrent studies, which included analyses of the relationship between student performance on the 
ADP Algebra II end-of-course exam and their performance on other exams, such as a state exam, the 
SAT or the ACT; 

■ Cross-sectional validity studies, in which Pearson administered the ADP Algebra II end-of-course exam 
to college students at the beginning of their first credit-bearing mathematics course and analyzed the 
relationship between the students’ final grades in the course and their performance on the exam; 

■ Judgment studies, in which college mathematics professors from both two- and four-year 
postsecondary institutions: a] rated the ADP Algebra II exam standards based on their relevancy to 
the courses they teach; b] determined which items on the ADP Algebra II end-of-course exam a student 
would need to have previously mastered in order to successfully complete their course, earning a B- or 
better; and c] provided descriptions of student performance that formed the basis of the achievement 
level definitions for the ADP Algebra II end-of-course exam. Altogether, over 125 mathematics faculty 
members from 79 institutions in 20 states participated in three judgment study meetings held 
during winter and spring 2009; and 
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■ College Algebra and Pre-Calculus course syllabi analysis, in which Achieve content specialists 

determined the relative importance of the content in the Algebra II exam standards for preparation for 
credit-bearing college mathematics courses, specifically College Algebra and Pre-Calculus. In all, 71 
higher education institutions and campuses, representing two- and four-year public institutions, across 
18 states contributed syllabi to the analysis. 

Results from all of the studies were presented to an Algebra II standard setting panel composed of 
representatives from the 15 Consortium states, including state education department policy, assessment and 
mathematics experts, as well as higher education mathematics professors from two- and four-year institutions. 
During a two-day meeting, the panelists engaged in rich discussions about the various pieces of research and 
the implications of each study’s results. At the conclusion of the meeting, the judgments of individual panelists 
were combined to produce a set of recommended achievement level cut scores for the ADP Algebra II end-of- 
course exam: " Well Prepared,' "Prepared or “Needs Preparation .” [For a fuller description of these performance 
levels, see the sidebar on page 9.) 



ADP Algebra I Exam Performance Levels 

Results for the ADP Algebra I Exam are reported according to four performance levels that 
indicate a student’s proficiency in Algebra I and are a useful indicator of his or her preparedness 
for higher-level mathematics courses. The knowledge, skills, and abilities required at each 
performance level and the scores required to achieve each level were defined in collaboration with 
state department mathematics specialists and high school mathematics teachers. 

ADVANCED The student consistently applies concepts, procedures, and skills needed to show 
mastery of Algebra I. The student is highly effective at devising and clearly communicating a 
wide range of strategies to solve complex mathematical and contextual problems. The student 
computes accurately and uses precise mathematical and symbolic language to solve problems 
and communicate solutions. The student’s explanations demonstrate the ability to use formal 
reasoning to justify solutions and evaluate the validity of solutions. 

PROFICIENT The student usually applies concepts, procedures, and skills to show adequate 
progress toward the mastery of Algebra I. The student is usually effective at devising and 
communicating a variety of strategies to solve mathematical and contextual problems. The 
student is adept in computation and uses mathematical and symbolic language to solve problems 
and communicate solutions. The student’s explanations demonstrate the ability to reason 
mathematically, recognizing connections between ideas in or across areas of mathematics, using 
formal and informal reasoning to justify solutions, and evaluating the validity of solutions. 

BASIC The student inconsistently applies concepts, procedures, and skills to show minimal 
progress toward the mastery of Algebra I. The student is generally effective at recalling and 
using routine, easily recognized, or straightforward strategies to solve simple mathematical 
and some contextual problems. The student can generally compute accurately and uses limited 
mathematical and symbolic language to solve problems and communicate solutions. The student’s 
explanations demonstrate limited ability to reason mathematically, using informal reasoning to 
justify solutions, and evaluating the validity of solutions. 

BELDW BASIC The student is unable to apply concepts, procedures, and skills to show progress 
toward the mastery of Algebra I. The student is usually unsuccessful at using problem solving 
strategies or uses inappropriate strategies to solve problems. The student’s explanations are 
often restatements of the problem, not related to the mathematics of the problem, or missing. 
The student displays limited computational accuracy. 
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ADP Algebra II Exam Performance Levels 

Results for the ADP Algebra II Exam are reported according to three performance levels that 
indicate a student’s proficiency in Algebra II and are a useful indicator of his or her preparedness 
for first-year credit-bearing college mathematics courses. The knowledge, skills, and abilities 
required at each performance level and the scores required to achieve each level were defined in 
collaboration with higher-education mathematics professors from across the country. 

WELL PREPARED The student consistently applies concepts, procedures, and skills needed to 
show mastery of Algebra II. The student is highly effective at devising and clearly communicating 
a wide range of strategies to solve complex mathematical and contextual problems. The 
student computes accurately and uses precise mathematical and symbolic language to solve 
problems and communicate solutions. The student’s explanations demonstrate the ability to 
reason mathematically, making appropriate connections between ideas in or across areas of 
mathematics, using formal reasoning to justify solutions, and evaluating the validity of solutions. 

PREPARED The student usually applies concepts, procedures, and skills to show adequate 
progress toward the mastery of Algebra II. The student is usually effective at devising and 
communicating a variety of strategies to solve mathematical and contextual problems. The 
student is adept in computation and uses mathematical and symbolic language to solve problems 
and communicate solutions. The student’s explanations demonstrate the ability to reason 
mathematically, recognizing connections between ideas in or across areas of mathematics, using 
formal and informal reasoning to justify solutions, and evaluating the validity of solutions. 

NEEDS PREPARATION The student inconsistently applies concepts, procedures, and skills to 
show minimal progress toward the mastery of Algebra II. The student is generally effective 
at recalling and using routine, easily recognized, or straightforward strategies to solve simple 
mathematical and some contextual problems. The student can generally compute accurately and 
uses limited mathematical and symbolic language to solve problems and communicate solutions. 
The student’s explanations demonstrate limited ability to reason mathematically, using informal 
reasoning to justify solutions, and evaluating the validity of solutions. 



Interpreting the 2009 Results 

Beginning with the 2009 exams, reporting scales have been established for the ADP Algebra I and II end-of- 
course exams that will support a standards-based interpretation of exam results and also allow comparisons 
of exam results across years. This is a critical requirement in the use of the exams to monitor states’ 
progress in implementing rigorous mathematics coursework and preparing their students for higher-level 
mathematics courses. The reports of exam results contain four key pieces of information: 

■ Performance levels; 

■ Scaled scores; 

■ Content standard mastery scores; and 

■ Number of students tested. 
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One important note of caution is in order. The ability to compare results across states is limited since the 
number and percentage of test takers varies significantly across the states (see Tables 3 and 4] and was not 
designed or intended by the states to be representative state samples. Because the factors for student 
selection were not controlled and the number of test takers varied significantly across the states, cross- 
state comparisons are not valid for the spring 2009 administration of either the Algebra I or II exam. It is 
expected that as states continue to expand their use of the test, the number of test takers will increase and 
the results will yield more comparable data. 

Performance Levels The ADP Algebra exams are reported primarily in terms of performance levels that 
classify student performance in relation to standards-based benchmarks established for each of the exams. 
State and consortium-wide results are reported as the percentage of students whose performance falls within 
each performance level category. The goal is for students to perform at the Proficient or Advanced levels on 
the Algebra I exam and at the Prepared or Well Prepared levels on the Algebra II exam. 

Scaled Scores In addition to performance levels, overall performance on the ADP Algebra exams is also 
reported as a scaled score. Each performance level category represents a range of performance, and scaled 
scores provide additional information describing where student performance falls within the performance 
level. Considered in conjunction with performance level results, scaled scores can be particularly useful in 
interpreting changes in exam results across years. State and Consortium scaled score results are reported 
as the mean and standard deviation of individual student scaled scores. 

Content Standard Mastery Scores Although the ADP Algebra exams are designed to support valid and 
reliable inferences at the overall content level, aggregate performance on each of the major standards 
can also provide information that may be useful in interpreting test results and improving curriculum and 
instruction. On the ADP Algebra exams, results for each of the major content standards are reported as the 
percent of students who reached the mastery level on that standard. The mastery level for each content 
standard is determined through a statistical process that compares student performance on the set of items 
within each content standard to the Proficient [for the ADP Algebra I exam] or Prepared [for the ADP Algebra 
II exam] performance standard established for the overall exam. Although results at the content standard 
level are much less reliable than results on the overall exam, they can be very useful. Aggregate results that 
show large and persistent differences in performance across content standards can be an indicator of relative 
strengths and weaknesses in curriculum and instruction. 

Number of Students Tested The total number and percentage of students tested is a critical piece of 
information in interpreting ADP exam results within and across states. As shown in Tables 3 and 4, the level 
of student participation varied greatly across the states in the Consortium. For example, in a few states, all 
students enrolled in an Algebra II course took the associated exam, and in others only a small subset of schools 
and students participated. The level of participation itself also can be an important indicator of progress in 
implementing the rigorous program of curriculum and instruction called for by the ADP Algebra I and II exam 
standards. It is expected that as states continue to align their curriculum and instruction with the exam 
standards, they will expand their use of the exams. 
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TABLE 3: STUDENTS' ENROLLMENT IN ALGEBRA I COURSES AND/OR PARTICIPATION IN ALGEBRA I 
EXAM BY STATE, 2008-09 



State 


Exam Participants 


Approximate Number 
of Students Enrolled in 
Algebra 1 


Percent of Enrolled 
Students who Took the 
Exam 


First Year Algebra 
1 is Required for 
Graduation 


Total 


33,446 


KY 


520 


54,160 


1% 


Currently Required 


NJ 


28,470 


105,000 


27% 


Currently Required 


OH 


2,031 


152,190 


1% 


2014 


Rl 


2,416 


No State Data 


No State Data 


Not Required 



NOTES: Data on the numbers of students enrolled in Algebra I are state reported information. Minnesota 
participated in the Algebra i administration but too few students took the exam to report scores. 

TABLE 4: STUDENTS' ENROLLMENT IN ALGEBRA II COURSES AND/OR PARTICIPATION IN ALGEBRA II 
EXAM BY STATE, 2008-09 



State 


Exam Participants 


Approximate Number 
of Students Enrolled 
in Algebra II (or its 
equivalent) 


Percent of Enrolled 
Students who Took the 
Exam 


First Year Algebra 
II is Required for 
Graduation 


Total 


102,936 


AZ 


2,982 


No State Data 


No State Data 


2013 


AR 


23,608 


29,119 


81% 


2010 


HI 


6,291 


7,266 


87% 


Not Required 


IN 


45,443 


60,078 


76% 


2011 


KY 


1,384 


45,350 


3% 


2012 


MD 


1,295 


No State Data 


No State Data 


Not Required 


MA 


584 


46,400 


1% 


Not Required 


MN 


1,164 


No State Data 


No State Data 


2015 


NJ 


8,063 


70,000 


12% 


Not Required 


NC 


2,551 


76,079 


3% 


2013 


OH 


2,416 


138,239 


2% 


2014 


PA 


6,786 


135,307 


5% 


Not Required 


Rl 


369 


No State Data 


No State Data 


Not Required 



NOTES: Data on the numbers of students enrolled in Algebra // are state reported information. Note that while 
Arkansas, Indiana and Hawaii required all students taking the Algebra it course to take the spring 2009 exam, 
discrepancies in numbers reported [course takers versus exam takers] are due to a variety of factors including 
students who completed their Algebra // course in a fall or winter semester or trimester; students who were 
absent during the testing window; students who enrolled in the class but did not complete it and; students 
repeating the course who were not retested. 
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ADP Algebra I and II Exam Results 

Students Performance was Low Across All States for Both Exams As shown in Table 5, performance on 
the 2009 Algebra I exam was low across all states. On the Algebra I exam, only 18.0% of the 33,446 students 
tested performed at the Proficient or Advanced levels. Additionally, in each of the participating states more 
than half of the students tested performed at the Below Basic level on the Algebra I test. 

TABLE 5: ALGEBRA I: PERCENT OF STUDENTS IN EACH PERFORMANCE LEVEL, BY STATE 





Total 


Advanced 


Proficient 


Basic 


Below Basic 


Average 
Scale Score 




Students 


[850-575] 


[574-450] 


[449-387] 


[386-300] 


[Standard 




Tested 










Deviation] 














384 


Consortium 


33,446 


1.6% 


16.4% 


26.2% 


55.8% 


(70) 














393 


KY 


520 


3.9% 


21.0% 


20.6% 


54.6% 


(84) 














386 


NJ 


28,470 


1.8% 


17.3% 


26.2% 


54.7% 


(72) 














375 


OH 


2,031 


0.4% 


13.0% 


26.4% 


60.2% 


(61) 














368 


Rl 


2,416 


0.2% 


8.2% 


27.2% 


64.4% 


(53) 



NOTE: Minnesota participated in the Algebra / administration but too few students took the exam to report 
scores. 

Table 6 shows that on the 2009 Algebra II exam, of the 102,936 students tested across 13 states, only 14.6% 
performed at the Prepared or Well Prepared\e\ie\s. Consortium wide, 85.4% of students tested performed at 
the Needs Preparation level. It is significant to note that the states with the highest participation rates who 
tested all students who were taking an Algebra II course in the spring of 2009— Arkansas, Indiana and Hawaii- 
had scores that were similar to those states who had a smaller, select number of participants. 
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TABLE 6: ALGEBRA II: PERCENT OF STUDENTS IN EACH PERFORMANCE LEVEL, BY STATE 





Total Students 


Well Prepared 


Prepared 


Needs 

Preparation 


Average Scale 
Score 




Tested 


(1650-1275) 


(1274-1150) 


(1149-900) 


(Standard 

Deviation] 


Consortium 


102,936 


3.5% 


11.1% 


85.4% 


1032 

(109) 


AZ 


2,982 


4.2% 


11.9% 


83.9% 


1045 

(110) 


AR 


23,608 


2.9% 


9.3% 


87.8% 


1019 

(105) 


HI 


6,291 


2.8% 


9.4% 


87.8% 


1021 

(105) 


IN 


45,443 


4.0% 


12.8% 


83.3% 


1043 

(110.5) 


KY 


1,384 


3.0% 


5.6% 


91 .4% 


1000 

(105) 


MD 


1,295 


2.9% 


12.4% 


84.7% 


1026 

(112) 


MA 


584 


4.1% 


14.9% 


81.1% 


1045 

(120) 


MN 


1,164 


0.9% 


5.2% 


93.9% 


1001 

(82) 


NJ 


8,063 


4.0% 


9.9% 


86.1% 


1026 

(113) 


NC 


2,551 


4.0% 


14.3% 


81.7% 


1055 

(108) 


OH 


2,416 


1.7% 


10.0% 


88.3% 


1030 

(94) 


PA 


6,786 


3.4% 


9.8% 


86.8% 


1025 

(108) 


Rl 


369 


1.9% 


10.8% 


87.3% 


1036 

(97) 
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The ADP Algebra I and II exam results are a stark reminder of the challenges states face to improve 
mathematics performance. The results, however, are not surprising given the rigor of the content the states 
chose to include in the exams and the performance standards which, by design, measure content mastery in 
the course they have just completed and preparation for higher-level mathematics coursework. In this regard, 
the exams are unique. 

The Algebra I exam, administered for the first time in 2009, is based on standards designed to ensure that 
students have the prerequisite content knowledge and skills needed to be prepared to perform successfully in 
higher-level mathematics course including an Algebra II course, aligned with the ADP Algebra II exam standards. 
Over the next several years of the program, as teachers and students become more familiar with the content 
and performance standards, and curriculum and instruction are designed to meet the higher expectations of 
those standards, it is expected that students will gain the necessary knowledge and skills needed to meet the 
standards and that will be reflected in improved exam scores. 

Similarly, the Algebra II exam is designed to measure mastery of advanced Algebra content at a level necessary 
to indicate readiness to perform successfully in a first-year credit-bearing college mathematics course. This is 
significantly more rigorous than what is measured by current statewide high school exams. As curriculum and 
instruction are designed to meet the ADP exam standards and coursework is more closely aligned with those 
standards, it is also expected that performance will improve. 

Moreover, as teachers and students become more familiar with the exam standards and those standards 
become more embedded in curriculum and instruction, student motivation— and, therefore, performance 
itself— on the ADP Algebra exams should increase. It is worth noting that student motivation may have 
played a role in this year’s results for both exams. There is no question that when there are no incentives or 
consequences associated with test results, 8 as is the case currently for both the ADP Algebra I and II end- 
of-course exams in all participating states, and the demands of the exam exceed their normal coursework, 
high school students may not be motivated to put forth their best effort. This does not fully explain low 
performance, but it is likely a contributing factor. 

Content Standard Mastery Results: Performance is Low Across All Content Standards in Both 
Exams, in All States Content standard mastery results provided in Tables 7 and 8 show that performance 
is consistently low across all of the major content standards. On the Algebra I test, the mastery level results 
ranged from 18.9% in Data, Statistics and Probabiiityto 26.5% in Non-linear Relationships. At the Consortium 
level, the percentage of students reaching the mastery level on the Algebra II content standards ranged only 
from 18.8% in Polynomials and Rational Functionsto 24.3% in Exponential Functions. Although patterns of 
performance varied somewhat across states, results were consistently low across the content standards. 

The results may suggest some content standards where performance was relatively stronger than others, 
but no content standard could be labeled a strength based on these initial results. And, as described in 
the previous section, these content standard mastery results based on small numbers of items are less 
reliable than the overall exam results and must be interpreted cautiously— particularly in states with non- 
representative samples of students tested. 



8 Brophy, J„ and Ames, C. [2005], NAEP Testing for Twelfth Graders: Motivational Issues. Washington, DC: National Assessment 
Governing Board. http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0OOD019b/80/3d/5f/2c.pdf 
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TABLE 7: ALGEBRA I: PERCENT OF STUDENTS AT MASTERY LEVEL,* BY STATE AND CONTENT 
STANDARD 





Total Tested 


Operations on 
Numbers and 
Expressions 


Linear 

Relationships 


Non-linear 

Relationships 


Data, Statistics 
and Probability 


Consortium 


33,446 


22.5% 


24.6% 


26.5% 


18.9% 


KY 


520 


23.9% 


27.1% 


32.1% 


25.2% 


NJ 


28,470 


23.8% 


25.3% 


27.7% 


19.5% 


OH 


2,031 


18.6% 


20.9% 


23.6% 


12.5% 


Rl 


2,416 


10.0% 


18.8% 


14.3% 


14.7% 



NOTE: Although Minnesota administered the 2009 ADP Algebra i Exam, state level data is not reported when 
too few students took the exam to report scores. 

TABLE 8: ALGEBRA II: PERCENT OF STUDENTS AT MASTERY LEVEL,* BY STATE AND CONTENT 
STANDARD 





Total Tested 


Operations on 
Numbers and 
Expressions 


Equations and 
Inequalities 


Polynomial 
and Rational 
Functions 


Exponential 

Functions 


Function 
Operations 
and Inverses 


Consortium 


102,936 


20.2% 


21.8% 


18.8% 


24.3% 


22.7% 


AZ 


2,982 


22.9% 


22.9% 


22.5% 


25.1% 


26.9% 


AR 


23,608 


13.9% 


18.9% 


17.9% 


22.4% 


21.3% 


HI 


6,291 


18.9% 


16.8% 


17.0% 


21.8% 


21.6% 


IN 


45,443 


24.8% 


24.6% 


19.7% 


26.3% 


23.2% 


KY 


1,384 


13.2% 


15.2% 


11.9% 


18.9% 


15.3% 


MD 


1,295 


16.3% 


20.3% 


18.8% 


27.2% 


25.4% 


MA 


584 


19.9% 


21.6% 


26.4% 


23.5% 


24.5% 


MN 


1,164 


7.9% 


12.4% 


12.6% 


18.9% 


16.8% 


NJ 


8,063 


19.2% 


20.3% 


16.9% 


23.0% 


22.1% 


NC 


2,551 


23.8% 


28.4% 


24.0% 


30.2% 


27.3% 


OH 


2,416 


16.6% 


19.5% 


18.1% 


26.7% 


23.1% 


PA 


6,786 


16.4% 


20.2% 


17.9% 


20.5% 


23.6% 


Rl 


369 


16.0% 


21.1% 


20.9% 


24.4% 


23.3% 



* The mastery le i /el for each content standard is determined through a statistical process that compares 
student performance on the set of items within each content standard to the Proficient [for the ADP Algebra i 
exam] or Prepared [for the ADP Algebra ii exam] performance standard established for the overall exam. 
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Consistent Patterns from 2008 to 2009 



Performance on Constructed Response Items Is Particularly Weak At least 30 percent of the points 
on the ADP Algebra I and II exams are derived from constructed-response items— 2-point short-answer and 
4-point extended-response items— that require a student to produce an answer rather than select from a 
set of four possible answers, as with a multiple-choice item. In general, the constructed-response format is 
well-suited for measuring critical higher-level knowledge and skills such as problem solving and reasoning. It is 
not unusual, therefore, for students to perform somewhat better overall on multiple-choice items than on the 
constructed-response items. However, it is not the case that all constructed-response items measure only 
high-level knowledge and skills or that the constructed-response items must measure more difficult content 
than the multiple-choice items. 

Although performance on the ADP Algebra exams is low across all item types, the low level of performance on 
the constructed-response items is particularly striking. On both the Algebra I and Algebra II exams, students 
earned, on average, only 11% and 14%, respectively, of the possible points available on constructed-response 
items. On the Algebra I exam, one-fourth of the students earned no points on the constructed-response items. 
Many of these students made no attempt to answer any of those constructed-response items. On the Algebra 
II exam in both 2008 and 2009, nearly one-third of the students earned no points on the 2-point or 4-point 
constructed-response items. Further, on the 2009 Algebra II exam more than half of the students earned no 
points on 4-point extended response items. 

Whether students performed poorly on the constructed-response items because they found the content to be 
too difficult or the task too demanding [relative to their motivation level, see previous discussion p. 14], remains 
a challenging issue for states to address. In the design of the ADP Algebra exams, the states agreed that the 
inclusion of constructed-response items on the tests was critical to send a clear signal that the goal was to 
produce curriculum and instructional programs that require students to demonstrate critical thinking and 
problem solving skills and communicate their reasoning and strategies. 

Students Who Take Algebra I and II in Earlier Grades Perform Better on the Exams As shown in Table 
9, students who completed Algebra I in 7th or 8th grade were much more likely to perform at the Proficient 
and Advanced performance levels than those students who completed Algebra I in grades 9-12. Consistent 
with results from the 2008 test, as shown in Table 10, students who take Algebra II in the 8th or 9th grade 
do better on the exam than those students who take the course in the 11th or 12th grade on average. Also 
consistent with the 2008 results, the performance gap between students who complete the course in grade 
10 compared to grade 1 1 is much greater than the gap between students who complete the course in grade 9 
compared to grade 10. This is most likely because students who are prepared to take the course in the earlier 
grades are the strongest and most advanced mathematics students, while those who take it toward the end of 
their high school career tend to struggle more in mathematics. 
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TABLE 9: ALGEBRA I: PERCENT OF STUDENTS IN EACH PERFORMANCE LEVEL, BY GRADE 





Total 

Students 

Tested 


Advanced 

(850-575) 


Proficient 

(574-450) 


Basic 

(449-387) 


Below Basic 
(386-300) 


Average 
Scale Score 

(Standard 

Deviation] 


Grade Seven 


584 


17.8% 


67.1% 


13.4% 


1.7% 


512 

(66) 


Grade Eight 


7,542 


4.9% 


38.2% 


31.5% 


25.4% 


437 

(77) 


Grade Nine 


19,926 


0.3% 


9.7% 


27.2% 


62.8% 


370 

(56) 


Grade Ten 


4,318 


0.4% 


5.7% 


16.4% 


77.6% 


351 

(53) 


Grade Eleven 


820 


0.1% 


3.9% 


16.8% 


79.2% 


348 

(48) 


Grade Twelve 


214 


0.0% 


1.9% 


16.8% 


81.3% 


343 

(42) 


Not Identified 


40 


0.0% 


0.0% 


17.5% 


82.5% 


341 

(40) 


Total 


33,446 


1.6% 


16.4% 


26.2% 


55.8% 


384 

(70) 



NOTE: Although there were Bth graders who took the 2009 ADP Algebra / exam, data are not reported when 
too few students took the exam to report scores. 
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TABLE 10: ALGEBRA II: PERCENT OF STUDENTS IN EACH PERFORMANCE LEVEL, BY GRADE 





Total Students 


Well Prepared 


Prepared 


Needs 

Preparation 


Average Scale 
Score 




Tested 


(1650-1275) 


(1274-1150) 


(1149-900) 


(Standard 

Deviation] 


Grade Eight 


212 


24.5% 


34.0% 


41.5% 


1195 

(141) 


Grade Nine 


7,115 


13.2% 


22.3% 


64.5% 


1111 

(138) 


Grade Ten 


32,079 


6.7% 


20.5% 


72.8% 


1080 

(117) 


Grade Eleven 


50,017 


0.7% 


5.8% 


93.5% 


1004 

(85) 


Grade Twelve 


13,390 


0.7% 


2.3% 


97.0% 


978 

(75) 


Not Identified 


118 


4.2% 


7.6% 


88.1% 


1025 

(111) 


Total 


102,936 


3.5% 


11.1% 


85.4% 


1032 

(109) 



NOTE: Although there were Bth and 7th graders who took the 2009 ADP Algebra II Exam, data are not 
reported when too few students took the exam to report scores. 
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Conclusion 



The ADP Assessment Consortium has succeeded in creating rigorous, high-quality assessments in Algebra 
I and II that will not only inform students whether they have mastered the content of the course they have 
just completed but also give an indication of preparation for higher-level mathematics courses. In 2009, the 
participating states gave an aligned Algebra I exam for the first time and administered the Algebra II exam 
to more students than they did in 2008. Given the significant pressure on state budgets this year, this is 
a significant accomplishment that reflects the Consortium states’ commitment to improving secondary 
mathematics so that all students graduate from high school prepared for the opportunities and challenges 
that await them. 

Now that rigorous exams with high standards are firmly in place, the next phase of work must begin in earnest. 
States must now turn their attention to the other purposes they identified in creating the exams: ensuring a 
consistent level of content and rigor within and across states in their Algebra I and II courses and improving 
the quality of curriculum and instruction in Algebra I and II courses, within and across states. To do so, states 
must provide students and teachers with the tools they need to be successful, including improving curriculum 
and instruction and providing professional development for teachers. Moreover, states must make sure all 
students have access to the support they need to succeed in rigorous mathematics courses. If states can be 
as successful on these critical next steps as they have been in creating the assessments, they will have truly 
changed the prospects for their students by ensuring that student choices regarding their future college and 
career plans are based on having a strong foundation in mathematics that makes all paths possible. 
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